With hosted email filtering services, outside client senders attempting to send a mail message to a SMTP (simple mail transport protocol) delivery system are instead directed to an email filtering service/server (hereinafter “filtering service” for brevity). The filtering service, commonly implemented as an SMTP relay or gateway, either accepts or rejects the message. For example, messages from known bad senders (e.g., “spammers”) may be rejected. If the message is accepted, the filtering service has responsibility for communicating the message to the delivery SMTP system.
The filtering service scans the message header and body in order to handle undesirable messages differently from other messages. In general, with respect to receiving messages, scanning them and delivery communications, such filtering services use either “proxy” semantics or “store-and-forward” semantics. Each type of semantics has benefits and drawbacks.
Proxy semantics operate over a TCP connection between the outside SMTP sending client and the filtering service, and a similar connection between the filtering service and the delivery system. A proxy-based filtering service processes (scans) messages and proxies communications back and forth to and from the delivery server and sender, without writing the message to disk or taking formal responsibility for the message. This is simple, inexpensive, facilitates high throughput, introduces relatively little latency for most messages, and avoids the need for sending bounce messages/NDRs (Non-Delivery Reports/Receipts) or quarantining messages.
However, with proxy semantics, the connection between the outside client and the filtering service has to be held open while scanning takes place; for slow connections and/or large messages (including messages with large attachments), this can be error-prone. Further, if a transient error occurs while attempting to deliver to the delivery system, the entire conversation needs to unwind and be retried later, including performing redundant scanning on the resubmitted message. Still further, when under heavy load, a proxy-based filtering service may not be able to handle received messages, and instead has to return a response (e.g., a 400-level response) to the outside sending clients that basically instructs those clients to try again later.
In contrast, store-and-forward type filtering services write each accepted input message to disk and then close the connection with the outside sender. Store-and-forward type filtering services then process the message as needed, and maintain it until the forwarding of the message to the delivery system is successful. This provides more predictable and reliable results in specific situations, such as when the delivery system (destination mail server) is unavailable, or when a message is particularly large. By storing messages, large messages including messages with large attachments can be analyzed without needing to hold open TCP connections. Further, transient errors may be overcome by re-attempting message sending until successful, without subjecting the message to redundant filtering. Still further, if the filtering service is under heavy load, the filtering service can still accept mail, which remains queued until the load lessens.
However, drawbacks to store-and-forward semantics include that storage is costly. Further, rejected mail needs to be handled somehow, as deleting the mail is counter to the SMTP specification, while NDR bounce notices that are sent are often mistaken as spam. Quarantining messages adds storage costs. Still further, there is more latency in end-to-end delivery, and there is a risk of mail loss if the filtering service crashes or fails after taking formal responsibility for a message, but before the message is delivered.